Link to part 1: https://uptrusting.com/post/LN01VP

Link to part 2: https://uptrusting.com/post/KPLe6Q

Link to part 3: https://uptrusting.com/post/lQlWYP

Note: Originally written for the participants of the AI alignment X spirituality/metaphysics retreats I’ve co-hosted with Jordan and Anna Salamon, so there may be some references to ideas or people you don’t know.

Steel-UpTrust as a facilitator for The Second Coming

At the heart of my conception of a positive singularity is AIs that help humanity align with what’s true and what’s good. What might such AIs actually look like? And how can we train the AIs to align with what’s true and good, when people seem to have irreconcilably different conceptions of truth and goodness?

I’ll start by describing how social media might look different with such AIs. Right now, AIs on social media are amplifying polarizing content and misinformation. They often show people in each filter bubble the worst content from everyone else’s filter bubbles, by the standards of their own filter bubble, thereby polarizing the filter bubbles further apart. If social media instead showed people the best content from everyone else’s filter bubbles, by the standards from within their own filter bubble, the filter bubbles would instead gradient toward coherence, leading to peace and mutual understanding. Truth-seeking, nuance, and peacemaking would be rewarded over attention-grabbing and polarization. (Unfortunately, I think current economic incentives don’t point in this direction – I think one of the biggest weaknesses of this vision is that it doesn’t provide a concrete story for how Molochian dynamics will actually get overcome in practice.)

Crucially, rather than a top-down enforcement of truth and goodness like we see in social media systems today (reminiscent of the oppression attractor; “orange” in Spiral Dynamics terms), or an absence of any enforcement (reminiscent of the chaos attractor; “green” in Spiral Dynamics terms), we start with people’s initial conceptions of what’s true and good, and allow for deeper forms of truth and goodness to emerge through the process of mutual understanding (third attractor; “teal” in Spiral Dynamics terms). This is one of the core ideas that came up when I was brainstorming with Jordan about his company UpTrust. I give Jordan a lot of credit for concretizing a lot of these thoughts, as I would probably not have been able to generate them on my own.

One could imagine generalizing the aforementioned core ideas beyond social media to pretty much any domain in which people care about discerning quality of some kind or another. Everybody would have an AI assistant that would help them make decisions in some domain of interest; the AIs would help their operators find trustworthy material, by leveraging the endorsements of people they trust, the people those people trust, and so on; and the incentives would be set up for these trust networks to eventually cohere. Such a system of AIs would play a crucial role in collective sensemaking as the world gets wilder and wilder, faster and faster, as basic assumptions about reality we’ve taken for granted slide out from under our feet. I’m calling such a system of AIs “Steel-UpTrust”.

An idealized end result of this coherence process would be a collective human superintelligence, a well-functioning superorganism with each participating human as a cell in its body. Whenever any cell in the superorganism has some kind of need, the need could get addressed in the most efficient possible way. If someone in Norway is dealing with a complex personal issue, just the right person in Kazakhstan might be able to help, and the connection would get facilitated.

Why would there be a “primary filter bubble” that all the existing filter bubbles cohere toward, instead of multiple different filter bubbles that simply fail to cohere? This is where I’m relying on the essence shared behind all religions – the claim that it’s ultimately in everybody’s best interests to treat everybody else’s joys and sufferings as their own. The process by which this truth gets revealed and embodied is the process by which the “Schelling filter bubble” / “Schelling coalition” coheres, and aligns with my conception of the unfolding of the Second Coming. Some components of this process that seem plausible to me:

Mass psychological healing
Mass recognition that insatiable materialistic desires (e.g. for money, fame, power, and status) ultimately ground out in desires for love and belonging
Mass recognition of the ethical and metaphysical truths shared across religions
Mass reassessments of history, including mass acknowledgments of historical injustices, according to the standards of these ethical truths (reminiscent of the Last Judgment)
Fading of attachments to tribal identities (like nation, race, and religion)
A new form of governance involving an upgraded form of democracy, in which individuals’ conflicting preferences are synthesized (as in the parable of the blind men and the elephant) to determine how society should be run, including how resources should be distributed
Phasing out of participation in dysfunctional systems (e.g. factory farming and environmental exploitation), and phasing in of participation in systems more aligned with life
Misguided spiritual leaders (“false prophets”) trying to claim unique spiritual authority, and genuine spiritual leaders (the vanguard of the Distributed Second Coming) who recognize and respect each others’ spiritual authority, collaborate very effectively, acknowledge and accept their own and each others’ personal shortcomings, and show compassion to the misguided spiritual leaders

It’s ultimately in everyone’s interests to opt into the Schelling filter bubble. As day-to-day life increasingly feels like an acid trip, with the dose increasing over the years, the Schelling filter bubble will remain robust and coherent, while most other filter bubbles will fracture and dissolve, as the false assumptions underlying their worldviews get revealed as incoherent.

Will everybody make it into the Schelling filter bubble? I don’t think so. I think some might just be happy not participating (e.g. uncontacted indigenous people). I think others might just resist the truth hard enough that they’ll never integrate into the Schelling filter bubble (perhaps some very committed young-earth creationists). It’s also plausible that entire nations (e.g. those run by inflexible autocrats) might prefer to wage war with the Schelling coalition than to integrate with it, at least toward the beginning, and that the Schelling coalition might itself have to wage a defensive war in order to triumph. (In charitable readings of the founding of Islam, the Prophet Muhammad pretty much did this in the context of his culture.)

I think the core technical challenges needed to realize this vision have little to do with AI per se. I think the hard part lies in designing the infrastructure for identifying and bridging the filter bubbles. In some sense, the core technical challenges here are roughly similar to the core technical challenges involved in:

designing an “exobrain” for humanity’s knowledge, trustworthy to the degree that formal verification is trustworthy, that can reliably integrate / translate between the same data expressed in different formats
designing infrastructure that would enable a Neuralinked world to function gracefully, rather than experiencing the same conflicts that are happening now, except at 10x the scale

I think the hardest part of the technical problem is the thicket of philosophical problems we must find technical solutions for (“what does it mean for two things to represent the same thing? what is a representation, anyway?”), and I think mathematical foundations for metaphysics (as described in a previous section) will play a crucial role in this solution.

It’s plausible to me that GPT-5 or GPT-6, in conjunction with this sort of infrastructure, could be sufficient to realize a large portion of the Steel-UpTrust vision. As an intuition pump, if mathematicians were just trying to effectively coordinate with each other to solve math problems (like with the Polymath Project), they could rely on formal verification to represent mathematical knowledge, and they can rely on AIs to help produce formally verified analogues of their proofs, to provide easily understandable natural language explanations of formally verified proofs, and to suggest possible places where mathematicians could help each other out. The abilities of AIs to do each of these things effectively will increase with the capabilities of the AI; the point isn’t for the AIs to be able to solve everything on their own.

My vision of Steel-UpTrust is very much like the one of mathematicians collaborating in the Polymath Project, except instead of solving math problems, the challenge at hand is figuring out how to live as human beings alongside other human beings, and instead of formal verification of math proofs, we’ll need interoperable formal representations of human know-how (including implicit, embodied knowledge).

How does Steel-UpTrust fit in with common existing conceptions of an AI singularity? In a world where superintelligent agentic consequentialist AIs are likely to emerge by default by training large neural networks, I think Steel-UpTrust is not very relevant, and I think we’re probably going to be screwed anyway. In a world where they don’t emerge by default from large neural networks, but could emerge from a new paradigm of AI, I think Steel-UpTrust would provide the groundwork for us to safely develop superintelligent AI.

In 2019, I had a conversation with Jordan about how the world is One Big Circle (*cough* relateful session *cough*), except it doesn’t have any facilitators. I basically think of Steel-UpTrust as providing facilitation for this One Big Circle, differentially amplifying the voices of truth and love until they prevail on earth.

#FutureYouLove

ai alignment

Second Coming <--> Positive Singularity <--> Steel-UpTrust? pt 4